Stor 455 Assignment #3
Michael Taffe
June 26, 2020
Part 1
mod1= lm(Price~LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath + BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms + Fireplaces + GarageCars + GarageSF + WoodDeckSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF, data=AmesTrain2)Best Subsets
all=regsubsets(Price~LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF + BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath + BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms + Fireplaces + GarageCars + GarageSF + WoodDeckSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF, data=AmesTrain2, nvmax= 22, nbest=2)
#ShowSubsets(all)The predictors that provide the lowest Mallow’s Cp together are LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.
StepWise
MSE=(summary(mod1)$sigma)^2
none=lm(Price~1,data=AmesTrain2)
step(none,scope=list(upper=mod1),scale=MSE)## Start: AIC=2973.27
## Price ~ 1
##
## Df Sum of Sq RSS Cp
## + GroundSF 1 1688573 1514775 1092.8
## + GarageCars 1 1469716 1733632 1336.7
## + GarageSF 1 1360709 1842639 1458.3
## + FirstSF 1 1281176 1922172 1546.9
## + BasementSF 1 1269388 1933960 1560.1
## + YearBuilt 1 1186095 2017254 1652.9
## + FullBath 1 1057617 2145731 1796.2
## + YearRemodel 1 1015342 2188006 1843.3
## + TotalRooms 1 824832 2378516 2055.7
## + Fireplaces 1 715821 2487528 2177.2
## + OpenPorchSF 1 498984 2704364 2419.0
## + BasementFinSF 1 409618 2793730 2518.6
## + HalfBath 1 368973 2834375 2563.9
## + SecondSF 1 294460 2908888 2647.0
## + LotArea 1 254497 2948851 2691.5
## + Bedroom 1 185248 3018100 2768.7
## + WoodDeckSF 1 164052 3039296 2792.4
## + BasementFBath 1 143206 3060142 2815.6
## + EnclosedPorchSF 1 120421 3082927 2841.0
## + LotFrontage 1 80830 3122518 2885.2
## + ScreenPorchSF 1 76363 3126985 2890.1
## <none> 3203348 2973.3
## + BasementHBath 1 1675 3201674 2973.4
##
## Step: AIC=1092.76
## Price ~ GroundSF
##
## Df Sum of Sq RSS Cp
## + YearBuilt 1 576252 938523 452.32
## + BasementSF 1 490057 1024718 548.41
## + GarageCars 1 392203 1122572 657.51
## + YearRemodel 1 384586 1130189 666.00
## + GarageSF 1 362417 1152358 690.71
## + FirstSF 1 350224 1164551 704.31
## + SecondSF 1 320201 1194574 737.78
## + BasementFinSF 1 235650 1279125 832.04
## + Bedroom 1 158563 1356212 917.98
## + BasementFBath 1 119542 1395233 961.48
## + Fireplaces 1 88525 1426250 996.06
## + EnclosedPorchSF 1 85220 1429555 999.75
## + FullBath 1 61884 1452891 1025.76
## + TotalRooms 1 57911 1456864 1030.19
## + LotArea 1 55466 1459309 1032.92
## + OpenPorchSF 1 42173 1472602 1047.74
## + LotFrontage 1 27156 1487619 1064.48
## + WoodDeckSF 1 18801 1495974 1073.79
## + ScreenPorchSF 1 17388 1497387 1075.37
## <none> 1514775 1092.76
## + HalfBath 1 358 1514417 1094.36
## + BasementHBath 1 76 1514699 1094.67
## - GroundSF 1 1688573 3203348 2973.27
##
## Step: AIC=452.32
## Price ~ GroundSF + YearBuilt
##
## Df Sum of Sq RSS Cp
## + BasementSF 1 196459 742064 235.29
## + FirstSF 1 168009 770514 267.01
## + SecondSF 1 159185 779338 276.85
## + BasementFinSF 1 115277 823246 325.80
## + GarageSF 1 94124 844399 349.38
## + LotArea 1 73208 865315 372.70
## + GarageCars 1 72790 865732 373.17
## + Bedroom 1 62989 875534 384.09
## + BasementFBath 1 57571 880952 390.13
## + Fireplaces 1 53422 885100 394.76
## + YearRemodel 1 51481 887042 396.92
## + LotFrontage 1 35938 902585 414.25
## + HalfBath 1 32034 906489 418.60
## + ScreenPorchSF 1 26084 912439 425.24
## + OpenPorchSF 1 11941 926582 441.01
## + TotalRooms 1 8387 930135 444.97
## + WoodDeckSF 1 4065 934458 449.79
## + FullBath 1 1873 936649 452.23
## <none> 938523 452.32
## + EnclosedPorchSF 1 1571 936952 452.57
## + BasementHBath 1 16 938506 454.30
## - YearBuilt 1 576252 1514775 1092.76
## - GroundSF 1 1078731 2017254 1652.95
##
## Step: AIC=235.29
## Price ~ GroundSF + YearBuilt + BasementSF
##
## Df Sum of Sq RSS Cp
## + YearRemodel 1 60923 681140 169.37
## + GarageSF 1 54583 687481 176.44
## + BasementFinSF 1 42909 699155 189.46
## + Bedroom 1 42382 699681 190.04
## + GarageCars 1 41002 701062 191.58
## + LotArea 1 33506 708558 199.94
## + Fireplaces 1 32163 709901 201.44
## + BasementFBath 1 21030 721034 213.85
## + LotFrontage 1 11533 730531 224.44
## + ScreenPorchSF 1 8187 733877 228.17
## + FirstSF 1 6690 735374 229.84
## + TotalRooms 1 4499 737564 232.28
## + SecondSF 1 4091 737972 232.73
## <none> 742064 235.29
## + OpenPorchSF 1 1683 740380 235.42
## + WoodDeckSF 1 1613 740451 235.50
## + EnclosedPorchSF 1 821 741243 236.38
## + BasementHBath 1 519 741545 236.72
## + FullBath 1 472 741592 236.77
## + HalfBath 1 59 742005 237.23
## - BasementSF 1 196459 938523 452.32
## - YearBuilt 1 282654 1024718 548.41
## - GroundSF 1 759473 1501537 1080.00
##
## Step: AIC=169.37
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel
##
## Df Sum of Sq RSS Cp
## + BasementFinSF 1 49265 631875 116.45
## + GarageSF 1 44306 636835 121.98
## + Fireplaces 1 39930 641210 126.86
## + LotArea 1 36176 644964 131.04
## + GarageCars 1 31199 649942 136.59
## + Bedroom 1 26888 654253 141.40
## + BasementFBath 1 19558 661582 149.57
## + LotFrontage 1 11393 669748 158.67
## + ScreenPorchSF 1 11132 670008 158.96
## + FullBath 1 6079 675061 164.60
## + FirstSF 1 5555 675585 165.18
## + TotalRooms 1 3520 677620 167.45
## + SecondSF 1 3245 677895 167.76
## <none> 681140 169.37
## + EnclosedPorchSF 1 1499 679642 169.70
## + HalfBath 1 716 680425 170.58
## + WoodDeckSF 1 511 680630 170.80
## + OpenPorchSF 1 134 681007 171.22
## + BasementHBath 1 38 681103 171.33
## - YearRemodel 1 60923 742064 235.29
## - YearBuilt 1 94413 775554 272.63
## - BasementSF 1 205902 887042 396.92
## - GroundSF 1 641846 1322987 882.94
##
## Step: AIC=116.45
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF
##
## Df Sum of Sq RSS Cp
## + GarageSF 1 45612 586263 67.599
## + GarageCars 1 34220 597655 80.299
## + LotArea 1 29908 601967 85.107
## + Fireplaces 1 24808 607067 90.793
## + Bedroom 1 17222 614653 99.249
## + LotFrontage 1 15008 616867 101.718
## + ScreenPorchSF 1 7190 624685 110.434
## + EnclosedPorchSF 1 3332 628543 114.735
## + FirstSF 1 2831 629044 115.293
## + FullBath 1 1908 629967 116.322
## <none> 631875 116.450
## + SecondSF 1 1475 630400 116.805
## + TotalRooms 1 906 630969 117.439
## + HalfBath 1 810 631065 117.546
## + BasementHBath 1 559 631316 117.826
## + OpenPorchSF 1 324 631551 118.089
## + WoodDeckSF 1 40 631835 118.405
## + BasementFBath 1 3 631872 118.447
## - BasementFinSF 1 49265 681140 169.373
## - YearRemodel 1 67280 699155 189.457
## - YearBuilt 1 81791 713666 205.635
## - BasementSF 1 128293 760168 257.478
## - GroundSF 1 648516 1280391 837.451
##
## Step: AIC=67.6
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF
##
## Df Sum of Sq RSS Cp
## + LotArea 1 28710 557554 37.592
## + Fireplaces 1 20760 565503 46.455
## + LotFrontage 1 8565 577698 60.050
## + Bedroom 1 8070 578193 60.602
## + ScreenPorchSF 1 5560 580704 63.401
## + HalfBath 1 2578 583685 66.725
## + EnclosedPorchSF 1 2154 584109 67.197
## <none> 586263 67.599
## + FullBath 1 1336 584927 68.110
## + BasementHBath 1 391 585872 69.163
## + FirstSF 1 373 585890 69.183
## + GarageCars 1 284 585980 69.283
## + TotalRooms 1 218 586045 69.356
## + WoodDeckSF 1 126 586138 69.459
## + BasementFBath 1 126 586138 69.459
## + OpenPorchSF 1 122 586142 69.463
## + SecondSF 1 102 586161 69.485
## - GarageSF 1 45612 631875 116.450
## - YearBuilt 1 49564 635827 120.856
## - BasementFinSF 1 50572 636835 121.979
## - YearRemodel 1 56391 642655 128.467
## - BasementSF 1 99767 686030 176.824
## - GroundSF 1 455152 1041415 573.028
##
## Step: AIC=37.59
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea
##
## Df Sum of Sq RSS Cp
## + Fireplaces 1 16178 541376 21.556
## + Bedroom 1 8660 548893 29.937
## + LotFrontage 1 6757 550797 32.059
## + ScreenPorchSF 1 5824 551730 33.099
## + HalfBath 1 3170 554384 36.058
## + EnclosedPorchSF 1 3072 554482 36.167
## <none> 557554 37.592
## + FullBath 1 1675 555879 37.724
## + BasementHBath 1 1408 556145 38.022
## + OpenPorchSF 1 472 557082 39.066
## + GarageCars 1 196 557358 39.373
## + TotalRooms 1 150 557404 39.425
## + FirstSF 1 83 557471 39.500
## + WoodDeckSF 1 35 557519 39.553
## + BasementFBath 1 35 557519 39.553
## + SecondSF 1 4 557550 39.588
## - LotArea 1 28710 586263 67.599
## - BasementFinSF 1 44314 601868 84.996
## - GarageSF 1 44413 601967 85.107
## - YearBuilt 1 56546 614100 98.633
## - YearRemodel 1 58424 615978 100.727
## - BasementSF 1 80372 637926 125.195
## - GroundSF 1 414038 971592 497.185
##
## Step: AIC=21.56
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces
##
## Df Sum of Sq RSS Cp
## + LotFrontage 1 6812 534564 15.961
## + Bedroom 1 6740 534636 16.042
## + ScreenPorchSF 1 3825 537550 19.291
## + EnclosedPorchSF 1 2894 538482 20.329
## + HalfBath 1 2453 538923 20.821
## <none> 541376 21.556
## + BasementHBath 1 1485 539890 21.900
## + FullBath 1 834 540541 22.626
## + OpenPorchSF 1 520 540855 22.976
## + WoodDeckSF 1 186 541189 23.348
## + SecondSF 1 159 541217 23.379
## + TotalRooms 1 97 541279 23.448
## + FirstSF 1 66 541310 23.482
## + BasementFBath 1 54 541321 23.495
## + GarageCars 1 6 541370 23.549
## - Fireplaces 1 16178 557554 37.592
## - LotArea 1 24128 565503 46.455
## - BasementFinSF 1 32717 574093 56.030
## - GarageSF 1 40938 582314 65.196
## - YearBuilt 1 53463 594839 79.159
## - YearRemodel 1 62705 604081 89.463
## - BasementSF 1 79162 620537 107.810
## - GroundSF 1 315742 857117 371.562
##
## Step: AIC=15.96
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage
##
## Df Sum of Sq RSS Cp
## + Bedroom 1 7280 527284 9.8451
## + ScreenPorchSF 1 4171 530393 13.3117
## + HalfBath 1 2825 531738 14.8116
## + EnclosedPorchSF 1 2246 532318 15.4578
## <none> 534564 15.9615
## + FullBath 1 994 533570 16.8533
## + BasementHBath 1 764 533800 17.1099
## + OpenPorchSF 1 407 534157 17.5075
## + SecondSF 1 336 534228 17.5873
## + TotalRooms 1 268 534296 17.6630
## + FirstSF 1 185 534379 17.7550
## + WoodDeckSF 1 31 534533 17.9273
## + GarageCars 1 11 534553 17.9498
## + BasementFBath 1 9 534555 17.9512
## - LotFrontage 1 6812 541376 21.5557
## - Fireplaces 1 16233 550797 32.0592
## - LotArea 1 22476 557040 39.0193
## - BasementFinSF 1 34709 569273 52.6574
## - GarageSF 1 35471 570035 53.5069
## - YearBuilt 1 57080 591643 77.5971
## - YearRemodel 1 63282 597845 84.5115
## - BasementSF 1 69635 604199 91.5945
## - GroundSF 1 317663 852227 368.1097
##
## Step: AIC=9.85
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom
##
## Df Sum of Sq RSS Cp
## + ScreenPorchSF 1 4063 523220 7.3153
## + HalfBath 1 2361 524922 9.2126
## <none> 527284 9.8451
## + EnclosedPorchSF 1 1693 525590 9.9573
## + TotalRooms 1 525 526759 11.2598
## + SecondSF 1 463 526820 11.3288
## + FullBath 1 438 526846 11.3571
## + OpenPorchSF 1 322 526961 11.4857
## + FirstSF 1 314 526970 11.4952
## + BasementHBath 1 203 527081 11.6193
## + WoodDeckSF 1 52 527231 11.7871
## + BasementFBath 1 26 527257 11.8157
## + GarageCars 1 0 527284 11.8451
## - Bedroom 1 7280 534564 15.9615
## - LotFrontage 1 7352 534636 16.0418
## - Fireplaces 1 14241 541525 23.7222
## - LotArea 1 23125 550408 33.6258
## - GarageSF 1 27990 555274 39.0500
## - BasementFinSF 1 29808 557091 41.0763
## - YearRemodel 1 53823 581106 67.8499
## - YearBuilt 1 59009 586292 73.6313
## - BasementSF 1 68154 595438 83.8272
## - GroundSF 1 241622 768905 277.2185
##
## Step: AIC=7.32
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom +
## ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## + EnclosedPorchSF 1 2268 520953 6.7871
## + HalfBath 1 1940 521281 7.1528
## <none> 523220 7.3153
## + TotalRooms 1 666 522554 8.5726
## + SecondSF 1 504 522717 8.7535
## + OpenPorchSF 1 397 522823 8.8727
## + FirstSF 1 361 522860 8.9132
## + FullBath 1 257 522963 9.0283
## + BasementHBath 1 250 522971 9.0368
## + BasementFBath 1 18 523202 9.2953
## + GarageCars 1 13 523208 9.3013
## + WoodDeckSF 1 2 523219 9.3136
## - ScreenPorchSF 1 4063 527284 9.8451
## - Bedroom 1 7173 530393 13.3117
## - LotFrontage 1 7702 530922 13.9015
## - Fireplaces 1 12356 535576 19.0901
## - LotArea 1 23509 546730 31.5247
## - GarageSF 1 27037 550257 35.4573
## - BasementFinSF 1 28213 551434 36.7691
## - YearRemodel 1 55235 578456 66.8948
## - YearBuilt 1 61090 584310 73.4218
## - BasementSF 1 63477 586697 76.0824
## - GroundSF 1 240344 763565 273.2643
##
## Step: AIC=6.79
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom +
## ScreenPorchSF + EnclosedPorchSF
##
## Df Sum of Sq RSS Cp
## + HalfBath 1 1795 519157 6.7855
## <none> 520953 6.7871
## - EnclosedPorchSF 1 2268 523220 7.3153
## + TotalRooms 1 606 520346 8.1112
## + OpenPorchSF 1 376 520577 8.3684
## + FullBath 1 281 520672 8.4737
## + SecondSF 1 244 520709 8.5155
## + BasementHBath 1 232 520721 8.5283
## + FirstSF 1 135 520817 8.6363
## + BasementFBath 1 32 520921 8.7514
## + GarageCars 1 17 520936 8.7681
## + WoodDeckSF 1 1 520952 8.7865
## - ScreenPorchSF 1 4637 525590 9.9573
## - Bedroom 1 6535 527488 12.0727
## - LotFrontage 1 7004 527957 12.5953
## - Fireplaces 1 12164 533116 18.3477
## - LotArea 1 24298 545251 31.8763
## - GarageSF 1 26545 547497 34.3805
## - BasementFinSF 1 29306 550258 37.4586
## - YearRemodel 1 56510 577463 67.7880
## - YearBuilt 1 62029 582982 73.9404
## - BasementSF 1 62342 583294 74.2891
## - GroundSF 1 233848 754801 265.4941
##
## Step: AIC=6.79
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom +
## ScreenPorchSF + EnclosedPorchSF + HalfBath
##
## Df Sum of Sq RSS Cp
## <none> 519157 6.7855
## - HalfBath 1 1795 520953 6.7871
## - EnclosedPorchSF 1 2123 521281 7.1528
## + TotalRooms 1 705 518452 7.9996
## + OpenPorchSF 1 205 518952 8.5572
## + BasementHBath 1 171 518986 8.5948
## + BasementFBath 1 41 519117 8.7403
## + GarageCars 1 16 519142 8.7682
## + FirstSF 1 15 519142 8.7686
## + FullBath 1 13 519145 8.7716
## + WoodDeckSF 1 0 519157 8.7855
## + SecondSF 1 0 519157 8.7855
## - ScreenPorchSF 1 4181 523339 9.4469
## - Bedroom 1 6174 525332 11.6691
## - LotFrontage 1 7291 526448 12.9136
## - Fireplaces 1 11778 530935 17.9161
## - LotArea 1 24678 543835 32.2977
## - GarageSF 1 27771 546929 35.7465
## - BasementFinSF 1 29712 548870 37.9104
## - YearBuilt 1 47897 567054 58.1832
## - YearRemodel 1 57616 576774 69.0193
## - BasementSF 1 61159 580317 72.9695
## - GroundSF 1 174283 693440 199.0855##
## Call:
## lm(formula = Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel +
## BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage +
## Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
##
## Coefficients:
## (Intercept) GroundSF YearBuilt BasementSF
## -2.126e+03 6.635e-02 4.613e-01 3.668e-02
## YearRemodel BasementFinSF GarageSF LotArea
## 6.174e-01 1.951e-02 4.164e-02 5.923e-04
## Fireplaces LotFrontage Bedroom ScreenPorchSF
## 8.402e+00 1.149e-01 -5.542e+00 4.200e-02
## EnclosedPorchSF HalfBath
## 3.737e-02 4.462e+00The best predictors together according to stepwise regression are LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.
Backward Elimination
MSE=(summary(mod1)$sigma)^2
step(mod1,scale=MSE)## Start: AIC=23
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath +
## BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms +
## Fireplaces + GarageCars + GarageSF + WoodDeckSF + OpenPorchSF +
## EnclosedPorchSF + ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## - GarageCars 1 1 517557 21.001
## - WoodDeckSF 1 9 517564 21.010
## - FullBath 1 84 517639 21.093
## - BasementFBath 1 167 517723 21.186
## - BasementHBath 1 201 517756 21.224
## - SecondSF 1 279 517835 21.311
## - FirstSF 1 283 517839 21.316
## - OpenPorchSF 1 357 517913 21.398
## - TotalRooms 1 704 518260 21.785
## - HalfBath 1 1022 518577 22.139
## - GroundSF 1 1296 518852 22.445
## <none> 517556 23.000
## - EnclosedPorchSF 1 2123 519679 23.367
## - ScreenPorchSF 1 4301 521857 25.795
## - Bedroom 1 5958 523514 27.642
## - LotFrontage 1 6472 524028 28.216
## - GarageSF 1 8606 526162 30.595
## - Fireplaces 1 10574 528130 32.789
## - BasementFinSF 1 19647 537203 42.904
## - LotArea 1 25105 542661 48.989
## - BasementSF 1 25690 543246 49.641
## - YearBuilt 1 42314 559870 68.174
## - YearRemodel 1 52874 570430 79.947
##
## Step: AIC=21
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath +
## BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms +
## Fireplaces + GarageSF + WoodDeckSF + OpenPorchSF + EnclosedPorchSF +
## ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## - WoodDeckSF 1 9 517566 19.011
## - FullBath 1 83 517640 19.093
## - BasementFBath 1 168 517725 19.189
## - BasementHBath 1 200 517757 19.225
## - SecondSF 1 280 517837 19.313
## - FirstSF 1 283 517840 19.317
## - OpenPorchSF 1 356 517913 19.398
## - TotalRooms 1 723 518280 19.807
## - HalfBath 1 1023 518580 20.142
## - GroundSF 1 1296 518853 20.446
## <none> 517557 21.001
## - EnclosedPorchSF 1 2122 519679 21.367
## - ScreenPorchSF 1 4303 521860 23.799
## - Bedroom 1 6038 523595 25.733
## - LotFrontage 1 6471 524028 26.216
## - Fireplaces 1 10696 528253 30.926
## - BasementFinSF 1 19649 537206 40.907
## - LotArea 1 25111 542668 46.997
## - GarageSF 1 25443 543000 47.367
## - BasementSF 1 25848 543405 47.818
## - YearBuilt 1 44031 561588 68.089
## - YearRemodel 1 52930 570487 78.011
##
## Step: AIC=19.01
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath +
## BasementHBath + FullBath + HalfBath + Bedroom + TotalRooms +
## Fireplaces + GarageSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## - FullBath 1 82 517648 17.103
## - BasementFBath 1 164 517730 17.194
## - BasementHBath 1 195 517761 17.229
## - SecondSF 1 278 517844 17.322
## - FirstSF 1 281 517847 17.325
## - OpenPorchSF 1 349 517915 17.400
## - TotalRooms 1 723 518289 17.817
## - HalfBath 1 1031 518597 18.161
## - GroundSF 1 1303 518869 18.464
## <none> 517566 19.011
## - EnclosedPorchSF 1 2114 519680 19.368
## - ScreenPorchSF 1 4309 521875 21.815
## - Bedroom 1 6056 523622 23.763
## - LotFrontage 1 6478 524044 24.233
## - Fireplaces 1 10792 528357 29.042
## - BasementFinSF 1 19709 537275 38.984
## - LotArea 1 25127 542693 45.025
## - GarageSF 1 25543 543109 45.489
## - BasementSF 1 25941 543507 45.932
## - YearBuilt 1 44075 561641 66.149
## - YearRemodel 1 53310 570876 76.445
##
## Step: AIC=17.1
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + FirstSF + SecondSF + GroundSF + BasementFBath +
## BasementHBath + HalfBath + Bedroom + TotalRooms + Fireplaces +
## GarageSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## - BasementFBath 1 133 517781 15.251
## - BasementHBath 1 201 517849 15.327
## - SecondSF 1 269 517917 15.402
## - FirstSF 1 275 517923 15.409
## - OpenPorchSF 1 333 517981 15.474
## - TotalRooms 1 671 518319 15.851
## - GroundSF 1 1283 518931 16.533
## - HalfBath 1 1352 519000 16.610
## <none> 517648 17.103
## - EnclosedPorchSF 1 2095 519743 17.439
## - ScreenPorchSF 1 4379 522026 19.984
## - Bedroom 1 6105 523753 21.909
## - LotFrontage 1 6456 524104 22.300
## - Fireplaces 1 10978 528626 27.342
## - BasementFinSF 1 19699 537346 37.064
## - LotArea 1 25062 542710 43.043
## - GarageSF 1 25793 543440 43.858
## - BasementSF 1 26108 543756 44.210
## - YearBuilt 1 48072 565720 68.696
## - YearRemodel 1 55056 572704 76.482
##
## Step: AIC=15.25
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + FirstSF + SecondSF + GroundSF + BasementHBath +
## HalfBath + Bedroom + TotalRooms + Fireplaces + GarageSF +
## OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## - BasementHBath 1 133 517914 13.399
## - SecondSF 1 258 518039 13.539
## - FirstSF 1 267 518048 13.549
## - OpenPorchSF 1 303 518084 13.589
## - TotalRooms 1 676 518457 14.005
## - GroundSF 1 1310 519092 14.712
## - HalfBath 1 1368 519149 14.777
## <none> 517781 15.251
## - EnclosedPorchSF 1 2072 519853 15.561
## - ScreenPorchSF 1 4382 522163 18.136
## - Bedroom 1 6039 523821 19.984
## - LotFrontage 1 6428 524209 20.418
## - Fireplaces 1 11006 528787 25.521
## - LotArea 1 24929 542710 41.044
## - BasementSF 1 26038 543819 42.280
## - GarageSF 1 26269 544050 42.538
## - BasementFinSF 1 30139 547920 46.852
## - YearBuilt 1 48039 565820 66.808
## - YearRemodel 1 54922 572704 74.482
##
## Step: AIC=13.4
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + FirstSF + SecondSF + GroundSF + HalfBath + Bedroom +
## TotalRooms + Fireplaces + GarageSF + OpenPorchSF + EnclosedPorchSF +
## ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## - SecondSF 1 251 518165 11.679
## - FirstSF 1 258 518172 11.687
## - OpenPorchSF 1 291 518205 11.723
## - TotalRooms 1 739 518653 12.223
## - GroundSF 1 1333 519247 12.885
## - HalfBath 1 1414 519328 12.976
## <none> 517914 13.399
## - EnclosedPorchSF 1 2076 519990 13.714
## - ScreenPorchSF 1 4348 522262 16.246
## - Bedroom 1 6590 524503 18.746
## - LotFrontage 1 6821 524734 19.003
## - Fireplaces 1 10963 528877 23.621
## - LotArea 1 24821 542735 39.071
## - BasementSF 1 26110 544024 40.508
## - GarageSF 1 26205 544119 40.614
## - BasementFinSF 1 30007 547921 44.853
## - YearBuilt 1 47936 565850 64.841
## - YearRemodel 1 55160 573074 72.895
##
## Step: AIC=11.68
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + FirstSF + GroundSF + HalfBath + Bedroom + TotalRooms +
## Fireplaces + GarageSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## - FirstSF 1 8 518173 9.6877
## - OpenPorchSF 1 272 518437 9.9820
## - TotalRooms 1 779 518944 10.5480
## - HalfBath 1 1551 519716 11.4087
## <none> 518165 11.6791
## - EnclosedPorchSF 1 2031 520196 11.9438
## - ScreenPorchSF 1 4376 522541 14.5575
## - Bedroom 1 6759 524924 17.2142
## - LotFrontage 1 6785 524950 17.2431
## - Fireplaces 1 11261 529426 22.2335
## - LotArea 1 25109 543274 37.6717
## - BasementSF 1 25923 544088 38.5794
## - GarageSF 1 27118 545283 39.9121
## - BasementFinSF 1 30176 548341 43.3214
## - YearBuilt 1 48699 566864 63.9714
## - YearRemodel 1 55175 573340 71.1913
## - GroundSF 1 95325 613490 115.9529
##
## Step: AIC=9.69
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + GroundSF + HalfBath + Bedroom + TotalRooms +
## Fireplaces + GarageSF + OpenPorchSF + EnclosedPorchSF + ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## - OpenPorchSF 1 280 518452 7.9996
## - TotalRooms 1 780 518952 8.5572
## - HalfBath 1 1700 519872 9.5828
## <none> 518173 9.6877
## - EnclosedPorchSF 1 2042 520215 9.9646
## - ScreenPorchSF 1 4395 522568 12.5880
## - Bedroom 1 6833 525006 15.3055
## - LotFrontage 1 6848 525021 15.3227
## - Fireplaces 1 11640 529813 20.6648
## - LotArea 1 25159 543332 35.7364
## - GarageSF 1 27475 545647 38.3181
## - BasementFinSF 1 30283 548455 41.4487
## - YearBuilt 1 48793 566966 62.0851
## - YearRemodel 1 55171 573343 69.1950
## - BasementSF 1 57499 575672 71.7913
## - GroundSF 1 103586 621759 123.1714
##
## Step: AIC=8
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + GroundSF + HalfBath + Bedroom + TotalRooms +
## Fireplaces + GarageSF + EnclosedPorchSF + ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## - TotalRooms 1 705 519157 6.7855
## <none> 518452 7.9996
## - HalfBath 1 1894 520346 8.1112
## - EnclosedPorchSF 1 2057 520509 8.2929
## - ScreenPorchSF 1 4309 522762 10.8039
## - Bedroom 1 6778 525231 13.5566
## - LotFrontage 1 6977 525430 13.7784
## - Fireplaces 1 11606 530058 18.9386
## - LotArea 1 24926 543378 33.7882
## - GarageSF 1 27731 546184 36.9161
## - BasementFinSF 1 30172 548625 39.6374
## - YearBuilt 1 48517 566969 60.0889
## - YearRemodel 1 56895 575347 69.4288
## - BasementSF 1 61263 579715 74.2991
## - GroundSF 1 108474 626927 126.9328
##
## Step: AIC=6.79
## Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel + BasementFinSF +
## BasementSF + GroundSF + HalfBath + Bedroom + Fireplaces +
## GarageSF + EnclosedPorchSF + ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## <none> 519157 6.7855
## - HalfBath 1 1795 520953 6.7871
## - EnclosedPorchSF 1 2123 521281 7.1528
## - ScreenPorchSF 1 4181 523339 9.4469
## - Bedroom 1 6174 525332 11.6691
## - LotFrontage 1 7291 526448 12.9136
## - Fireplaces 1 11778 530935 17.9161
## - LotArea 1 24678 543835 32.2977
## - GarageSF 1 27771 546929 35.7465
## - BasementFinSF 1 29712 548870 37.9104
## - YearBuilt 1 47897 567054 58.1832
## - YearRemodel 1 57616 576774 69.0193
## - BasementSF 1 61159 580317 72.9695
## - GroundSF 1 174283 693440 199.0855##
## Call:
## lm(formula = Price ~ LotFrontage + LotArea + YearBuilt + YearRemodel +
## BasementFinSF + BasementSF + GroundSF + HalfBath + Bedroom +
## Fireplaces + GarageSF + EnclosedPorchSF + ScreenPorchSF,
## data = AmesTrain2)
##
## Coefficients:
## (Intercept) LotFrontage LotArea YearBuilt
## -2.126e+03 1.149e-01 5.923e-04 4.613e-01
## YearRemodel BasementFinSF BasementSF GroundSF
## 6.174e-01 1.951e-02 3.668e-02 6.635e-02
## HalfBath Bedroom Fireplaces GarageSF
## 4.462e+00 -5.542e+00 8.402e+00 4.164e-02
## EnclosedPorchSF ScreenPorchSF
## 3.737e-02 4.200e-02The best predictors together according to Backward Elimination are LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.
Forward Selection
none=lm(Price~1,data=AmesTrain2)
step(none,scope=list(upper=mod1),scale=MSE, direction="forward")## Start: AIC=2973.27
## Price ~ 1
##
## Df Sum of Sq RSS Cp
## + GroundSF 1 1688573 1514775 1092.8
## + GarageCars 1 1469716 1733632 1336.7
## + GarageSF 1 1360709 1842639 1458.3
## + FirstSF 1 1281176 1922172 1546.9
## + BasementSF 1 1269388 1933960 1560.1
## + YearBuilt 1 1186095 2017254 1652.9
## + FullBath 1 1057617 2145731 1796.2
## + YearRemodel 1 1015342 2188006 1843.3
## + TotalRooms 1 824832 2378516 2055.7
## + Fireplaces 1 715821 2487528 2177.2
## + OpenPorchSF 1 498984 2704364 2419.0
## + BasementFinSF 1 409618 2793730 2518.6
## + HalfBath 1 368973 2834375 2563.9
## + SecondSF 1 294460 2908888 2647.0
## + LotArea 1 254497 2948851 2691.5
## + Bedroom 1 185248 3018100 2768.7
## + WoodDeckSF 1 164052 3039296 2792.4
## + BasementFBath 1 143206 3060142 2815.6
## + EnclosedPorchSF 1 120421 3082927 2841.0
## + LotFrontage 1 80830 3122518 2885.2
## + ScreenPorchSF 1 76363 3126985 2890.1
## <none> 3203348 2973.3
## + BasementHBath 1 1675 3201674 2973.4
##
## Step: AIC=1092.76
## Price ~ GroundSF
##
## Df Sum of Sq RSS Cp
## + YearBuilt 1 576252 938523 452.32
## + BasementSF 1 490057 1024718 548.41
## + GarageCars 1 392203 1122572 657.51
## + YearRemodel 1 384586 1130189 666.00
## + GarageSF 1 362417 1152358 690.71
## + FirstSF 1 350224 1164551 704.31
## + SecondSF 1 320201 1194574 737.78
## + BasementFinSF 1 235650 1279125 832.04
## + Bedroom 1 158563 1356212 917.98
## + BasementFBath 1 119542 1395233 961.48
## + Fireplaces 1 88525 1426250 996.06
## + EnclosedPorchSF 1 85220 1429555 999.75
## + FullBath 1 61884 1452891 1025.76
## + TotalRooms 1 57911 1456864 1030.19
## + LotArea 1 55466 1459309 1032.92
## + OpenPorchSF 1 42173 1472602 1047.74
## + LotFrontage 1 27156 1487619 1064.48
## + WoodDeckSF 1 18801 1495974 1073.79
## + ScreenPorchSF 1 17388 1497387 1075.37
## <none> 1514775 1092.76
## + HalfBath 1 358 1514417 1094.36
## + BasementHBath 1 76 1514699 1094.67
##
## Step: AIC=452.32
## Price ~ GroundSF + YearBuilt
##
## Df Sum of Sq RSS Cp
## + BasementSF 1 196459 742064 235.29
## + FirstSF 1 168009 770514 267.01
## + SecondSF 1 159185 779338 276.85
## + BasementFinSF 1 115277 823246 325.80
## + GarageSF 1 94124 844399 349.38
## + LotArea 1 73208 865315 372.70
## + GarageCars 1 72790 865732 373.17
## + Bedroom 1 62989 875534 384.09
## + BasementFBath 1 57571 880952 390.13
## + Fireplaces 1 53422 885100 394.76
## + YearRemodel 1 51481 887042 396.92
## + LotFrontage 1 35938 902585 414.25
## + HalfBath 1 32034 906489 418.60
## + ScreenPorchSF 1 26084 912439 425.24
## + OpenPorchSF 1 11941 926582 441.01
## + TotalRooms 1 8387 930135 444.97
## + WoodDeckSF 1 4065 934458 449.79
## + FullBath 1 1873 936649 452.23
## <none> 938523 452.32
## + EnclosedPorchSF 1 1571 936952 452.57
## + BasementHBath 1 16 938506 454.30
##
## Step: AIC=235.29
## Price ~ GroundSF + YearBuilt + BasementSF
##
## Df Sum of Sq RSS Cp
## + YearRemodel 1 60923 681140 169.37
## + GarageSF 1 54583 687481 176.44
## + BasementFinSF 1 42909 699155 189.46
## + Bedroom 1 42382 699681 190.04
## + GarageCars 1 41002 701062 191.58
## + LotArea 1 33506 708558 199.94
## + Fireplaces 1 32163 709901 201.44
## + BasementFBath 1 21030 721034 213.85
## + LotFrontage 1 11533 730531 224.44
## + ScreenPorchSF 1 8187 733877 228.17
## + FirstSF 1 6690 735374 229.84
## + TotalRooms 1 4499 737564 232.28
## + SecondSF 1 4091 737972 232.73
## <none> 742064 235.29
## + OpenPorchSF 1 1683 740380 235.42
## + WoodDeckSF 1 1613 740451 235.50
## + EnclosedPorchSF 1 821 741243 236.38
## + BasementHBath 1 519 741545 236.72
## + FullBath 1 472 741592 236.77
## + HalfBath 1 59 742005 237.23
##
## Step: AIC=169.37
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel
##
## Df Sum of Sq RSS Cp
## + BasementFinSF 1 49265 631875 116.45
## + GarageSF 1 44306 636835 121.98
## + Fireplaces 1 39930 641210 126.86
## + LotArea 1 36176 644964 131.04
## + GarageCars 1 31199 649942 136.59
## + Bedroom 1 26888 654253 141.40
## + BasementFBath 1 19558 661582 149.57
## + LotFrontage 1 11393 669748 158.67
## + ScreenPorchSF 1 11132 670008 158.96
## + FullBath 1 6079 675061 164.60
## + FirstSF 1 5555 675585 165.18
## + TotalRooms 1 3520 677620 167.45
## + SecondSF 1 3245 677895 167.76
## <none> 681140 169.37
## + EnclosedPorchSF 1 1499 679642 169.70
## + HalfBath 1 716 680425 170.58
## + WoodDeckSF 1 511 680630 170.80
## + OpenPorchSF 1 134 681007 171.22
## + BasementHBath 1 38 681103 171.33
##
## Step: AIC=116.45
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF
##
## Df Sum of Sq RSS Cp
## + GarageSF 1 45612 586263 67.599
## + GarageCars 1 34220 597655 80.299
## + LotArea 1 29908 601967 85.107
## + Fireplaces 1 24808 607067 90.793
## + Bedroom 1 17222 614653 99.249
## + LotFrontage 1 15008 616867 101.718
## + ScreenPorchSF 1 7190 624685 110.434
## + EnclosedPorchSF 1 3332 628543 114.735
## + FirstSF 1 2831 629044 115.293
## + FullBath 1 1908 629967 116.322
## <none> 631875 116.450
## + SecondSF 1 1475 630400 116.805
## + TotalRooms 1 906 630969 117.439
## + HalfBath 1 810 631065 117.546
## + BasementHBath 1 559 631316 117.826
## + OpenPorchSF 1 324 631551 118.089
## + WoodDeckSF 1 40 631835 118.405
## + BasementFBath 1 3 631872 118.447
##
## Step: AIC=67.6
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF
##
## Df Sum of Sq RSS Cp
## + LotArea 1 28709.5 557554 37.592
## + Fireplaces 1 20760.0 565503 46.455
## + LotFrontage 1 8564.8 577698 60.050
## + Bedroom 1 8070.2 578193 60.602
## + ScreenPorchSF 1 5559.7 580704 63.401
## + HalfBath 1 2577.8 583685 66.725
## + EnclosedPorchSF 1 2154.2 584109 67.197
## <none> 586263 67.599
## + FullBath 1 1335.8 584927 68.110
## + BasementHBath 1 391.2 585872 69.163
## + FirstSF 1 373.3 585890 69.183
## + GarageCars 1 283.8 585980 69.283
## + TotalRooms 1 217.9 586045 69.356
## + WoodDeckSF 1 125.8 586138 69.459
## + BasementFBath 1 125.6 586138 69.459
## + OpenPorchSF 1 121.6 586142 69.463
## + SecondSF 1 102.5 586161 69.485
##
## Step: AIC=37.59
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea
##
## Df Sum of Sq RSS Cp
## + Fireplaces 1 16178.1 541376 21.556
## + Bedroom 1 8660.4 548893 29.937
## + LotFrontage 1 6756.8 550797 32.059
## + ScreenPorchSF 1 5824.1 551730 33.099
## + HalfBath 1 3170.0 554384 36.058
## + EnclosedPorchSF 1 3072.2 554482 36.167
## <none> 557554 37.592
## + FullBath 1 1675.2 555879 37.724
## + BasementHBath 1 1408.3 556145 38.022
## + OpenPorchSF 1 471.8 557082 39.066
## + GarageCars 1 196.2 557358 39.373
## + TotalRooms 1 149.8 557404 39.425
## + FirstSF 1 82.5 557471 39.500
## + WoodDeckSF 1 34.8 557519 39.553
## + BasementFBath 1 34.6 557519 39.553
## + SecondSF 1 3.8 557550 39.588
##
## Step: AIC=21.56
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces
##
## Df Sum of Sq RSS Cp
## + LotFrontage 1 6811.8 534564 15.961
## + Bedroom 1 6739.8 534636 16.042
## + ScreenPorchSF 1 3825.2 537550 19.291
## + EnclosedPorchSF 1 2893.9 538482 20.329
## + HalfBath 1 2452.7 538923 20.821
## <none> 541376 21.556
## + BasementHBath 1 1485.2 539890 21.900
## + FullBath 1 834.2 540541 22.626
## + OpenPorchSF 1 520.2 540855 22.976
## + WoodDeckSF 1 186.5 541189 23.348
## + SecondSF 1 158.9 541217 23.379
## + TotalRooms 1 96.5 541279 23.448
## + FirstSF 1 66.0 541310 23.482
## + BasementFBath 1 54.4 541321 23.495
## + GarageCars 1 5.7 541370 23.549
##
## Step: AIC=15.96
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage
##
## Df Sum of Sq RSS Cp
## + Bedroom 1 7280.2 527284 9.8451
## + ScreenPorchSF 1 4170.7 530393 13.3117
## + HalfBath 1 2825.4 531738 14.8116
## + EnclosedPorchSF 1 2245.8 532318 15.4578
## <none> 534564 15.9615
## + FullBath 1 994.0 533570 16.8533
## + BasementHBath 1 763.9 533800 17.1099
## + OpenPorchSF 1 407.3 534157 17.5075
## + SecondSF 1 335.7 534228 17.5873
## + TotalRooms 1 267.8 534296 17.6630
## + FirstSF 1 185.2 534379 17.7550
## + WoodDeckSF 1 30.7 534533 17.9273
## + GarageCars 1 10.5 534553 17.9498
## + BasementFBath 1 9.3 534555 17.9512
##
## Step: AIC=9.85
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom
##
## Df Sum of Sq RSS Cp
## + ScreenPorchSF 1 4063.1 523220 7.3153
## + HalfBath 1 2361.3 524922 9.2126
## <none> 527284 9.8451
## + EnclosedPorchSF 1 1693.4 525590 9.9573
## + TotalRooms 1 525.0 526759 11.2598
## + SecondSF 1 463.1 526820 11.3288
## + FullBath 1 437.7 526846 11.3571
## + OpenPorchSF 1 322.4 526961 11.4857
## + FirstSF 1 313.9 526970 11.4952
## + BasementHBath 1 202.5 527081 11.6193
## + WoodDeckSF 1 52.1 527231 11.7871
## + BasementFBath 1 26.3 527257 11.8157
## + GarageCars 1 0.0 527284 11.8451
##
## Step: AIC=7.32
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom +
## ScreenPorchSF
##
## Df Sum of Sq RSS Cp
## + EnclosedPorchSF 1 2267.69 520953 6.7871
## + HalfBath 1 1939.71 521281 7.1528
## <none> 523220 7.3153
## + TotalRooms 1 666.18 522554 8.5726
## + SecondSF 1 503.87 522717 8.7535
## + OpenPorchSF 1 397.00 522823 8.8727
## + FirstSF 1 360.63 522860 8.9132
## + FullBath 1 257.43 522963 9.0283
## + BasementHBath 1 249.76 522971 9.0368
## + BasementFBath 1 17.94 523202 9.2953
## + GarageCars 1 12.58 523208 9.3013
## + WoodDeckSF 1 1.52 523219 9.3136
##
## Step: AIC=6.79
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom +
## ScreenPorchSF + EnclosedPorchSF
##
## Df Sum of Sq RSS Cp
## + HalfBath 1 1795.41 519157 6.7855
## <none> 520953 6.7871
## + TotalRooms 1 606.31 520346 8.1112
## + OpenPorchSF 1 375.57 520577 8.3684
## + FullBath 1 281.18 520672 8.4737
## + SecondSF 1 243.62 520709 8.5155
## + BasementHBath 1 232.20 520721 8.5283
## + FirstSF 1 135.33 520817 8.6363
## + BasementFBath 1 32.02 520921 8.7514
## + GarageCars 1 17.04 520936 8.7681
## + WoodDeckSF 1 0.57 520952 8.7865
##
## Step: AIC=6.79
## Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel + BasementFinSF +
## GarageSF + LotArea + Fireplaces + LotFrontage + Bedroom +
## ScreenPorchSF + EnclosedPorchSF + HalfBath
##
## Df Sum of Sq RSS Cp
## <none> 519157 6.7855
## + TotalRooms 1 704.95 518452 7.9996
## + OpenPorchSF 1 204.81 518952 8.5572
## + BasementHBath 1 171.06 518986 8.5948
## + BasementFBath 1 40.59 519117 8.7403
## + GarageCars 1 15.56 519142 8.7682
## + FirstSF 1 15.13 519142 8.7686
## + FullBath 1 12.50 519145 8.7716
## + WoodDeckSF 1 0.04 519157 8.7855
## + SecondSF 1 0.01 519157 8.7855##
## Call:
## lm(formula = Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel +
## BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage +
## Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
##
## Coefficients:
## (Intercept) GroundSF YearBuilt BasementSF
## -2.126e+03 6.635e-02 4.613e-01 3.668e-02
## YearRemodel BasementFinSF GarageSF LotArea
## 6.174e-01 1.951e-02 4.164e-02 5.923e-04
## Fireplaces LotFrontage Bedroom ScreenPorchSF
## 8.402e+00 1.149e-01 -5.542e+00 4.200e-02
## EnclosedPorchSF HalfBath
## 3.737e-02 4.462e+00The best predictors together according to Forward Selection are LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.
After using the methods of Best subsets, stepwise regression, backward elimination, and forward selection, we have determined that the best model to predict Price uses the following predictors: LotFrontage, LotArea, YearBuilt, YearRemodel, BasementFinSF, BasementSF, GroundSF, HalfBath, Bedroom, Fireplaces, GarageSF, EnclosedPorchSF, and ScreenPorchSF.
Every method gave us these predictors as the best predictors to use for a model because these predictors had the lowest Mallow’s Cp. The lower the Mallow’s Cp, the better the compromise between having little error in the model and not having too many predictors. In this case, this combination of predictors gave a Mallow’s Cp of 6.79, which was the lowest possible Mallow’s Cp given the predictors.
Here’s the summary for our model:
mod2= lm(Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel +
BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage +
Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
summary(mod2)##
## Call:
## lm(formula = Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel +
## BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage +
## Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
##
## Residuals:
## Min 1Q Median 3Q Max
## -148.184 -16.828 -1.793 14.028 183.353
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -2.126e+03 1.465e+02 -14.509 < 2e-16 ***
## GroundSF 6.635e-02 4.731e-03 14.026 < 2e-16 ***
## YearBuilt 4.613e-01 6.273e-02 7.353 6.57e-13 ***
## BasementSF 3.668e-02 4.415e-03 8.309 6.75e-16 ***
## YearRemodel 6.174e-01 7.655e-02 8.064 4.17e-15 ***
## BasementFinSF 1.951e-02 3.368e-03 5.791 1.14e-08 ***
## GarageSF 4.164e-02 7.437e-03 5.599 3.32e-08 ***
## LotArea 5.923e-04 1.122e-04 5.278 1.84e-07 ***
## Fireplaces 8.402e+00 2.304e+00 3.646 0.00029 ***
## LotFrontage 1.149e-01 4.004e-02 2.869 0.00427 **
## Bedroom -5.542e+00 2.099e+00 -2.640 0.00851 **
## ScreenPorchSF 4.200e-02 1.933e-02 2.172 0.03022 *
## EnclosedPorchSF 3.737e-02 2.414e-02 1.548 0.12212
## HalfBath 4.462e+00 3.135e+00 1.424 0.15510
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 29.76 on 586 degrees of freedom
## Multiple R-squared: 0.8379, Adjusted R-squared: 0.8343
## F-statistic: 233.1 on 13 and 586 DF, p-value: < 2.2e-16The following predictors are not significant at a 5% level in this model: EnclosedPorchSF and HalfBath. However, this does not necessarily mean they are bad predictors, as their higher p-values may be inflated due to multicollinearity. Additionally, their p-values are both still below .2, so while they are not significant at a 5% level, that does not necessarily mean they are bad predictors.
Let’s look into the VIF values:
vif(mod2)## GroundSF YearBuilt BasementSF YearRemodel
## 3.359649 2.588815 1.949760 1.773424
## BasementFinSF GarageSF LotArea Fireplaces
## 1.289299 1.832199 1.126276 1.394249
## LotFrontage Bedroom ScreenPorchSF EnclosedPorchSF
## 1.090803 1.787478 1.090471 1.278974
## HalfBath
## 1.700918VIF is used to detect multicollinearity. A VIF value greater than 5 generally causes concern for substantial multicollinearity, but a value of around 2 or 3 to 5 can still cause concern for moderate multicollinearity. No predictors have a VIF value greater than 5, but there are 2 predictors with VIF values greater than 2.5 that cause concern for moderate multicollinearity. These predictors are GroundSF and YearBuilt. The rest of the predictors have VIF values of between 1 and 2, causing little concern for multicollinearity.
Part 2
First we’ll check for linearity
plot(mod2)
hist(mod2$residuals)
Looking at the residuals vs fitted values plot, the linearity condition does not seem to be fully met. There appears to be curvature of the residuals, as it dips down and then curves up, suggesting that linearity is not met.
The zero mean condition seems to be met somewhat well, as the residuals seem to be generally spread around the zero line in the residuals vs. fitted values plot. The zero line seems to be close to passing through the center of the data, but it doesn’t quite do so perfectly.
The constant variance condition does not appear to be met, as there is definitely greater variance as fitted values reach around 250 and beyond than before they reach 250. The data is definitely more clustered in the middle of the plot, and there is also some curvature.
Looking at the normal q-q plot and the histogram, the independence condition appears to be reasonably met. In the normal q-q plot, the points generally follow the line for the most part, although they trail off a little at both tails. The histogram has a general bell curve and does not have a strong skew, further suggesting that the independence condition is met.
Checking for unusual studentized residuals
which(rstudent(mod2) > 3)## 62 70 198 202 374 537 572 581
## 62 70 198 202 374 537 572 581The criteria we are using to identify “unsual” cases for studentized residuals is if the studentized residual is greater than 3, then it is “unusual.” Based on this criteria, the following indices are unsusual in terms of studentized residuals: 62, 70 198, 202, 374, 537, 572, 581. It is important to note that just because these points are unusual in terms of their studentized residuals doesn’t mean that they are influential.
Checking for influential points
To check to see if these points are influential, we’ll see if the studentized residuals are different or similar to the standardized residuals. If they are similar for a point, the point is not very influential.
rstudent(mod2)[c(62, 70, 198, 202, 374, 537, 572, 581)]- rstandard(mod2)[c(62, 70, 198, 202, 374, 537, 572, 581)]## 62 70 198 202 374 537
## 0.05457370 0.02257026 0.23663538 0.02943293 0.07009602 0.04625260
## 572 581
## 0.05869388 0.02076641None of these points appear to be very influential, as the studentized residuals appear to be similar to the standardized residuals. The point that appears to be most influential is index 198, as the difference is .24. However, this still does not warrant it being called very influential. In case we missed anything, let’s see if there are any points that had a greater difference in studentized and standardized residuals than index 198.
which.max(rstudent(mod2)-rstandard(mod2))## 198
## 198It appears that 198 has the greatest difference in studentized and standardized residuals, suggesting that it is the most influential point. However, this point is not very influential. As one final check for influential points, we’ll return to the plot showing Cook’s distance to see if it shows any influential points.
plot(mod2, 5)
None of the points are outside of a Cook’s Distance of 1, or even 0.5, suggesting that no points are substantially influential.
Because no points are substantially influential and all four methods of determining predictors gave us our current predictors, we will not be making any changes to our data or model in Part 2.
Part 3
Transformations
The following transformations were tested, with mod3 proving to be the best transformation. mod3 transforms the response with a log function. We decided to try a logarithmic transformation to try to help with the constant variance and curvature/ lack of linearity issue we had originally.
mod3= lm(log(Price) ~ GroundSF + YearBuilt + BasementSF + YearRemodel +
BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage +
Bedroom + ScreenPorchSF + EnclosedPorchSF + HalfBath, data = AmesTrain2)
plot(mod3)
mod4= lm((Price)^2 ~ I(GroundSF^2) + I(YearBuilt^2) + I(BasementSF^2) + I(YearRemodel^2) + I(BasementFinSF^2) + I(GarageSF^2) + I(LotArea^2) + I(Fireplaces^2) + I(LotFrontage^2) + I(Bedroom^2) + I(ScreenPorchSF^2) + I(EnclosedPorchSF^2) + I(HalfBath^2), data = AmesTrain2)
plot(mod4)
mod5= lm((Price)^(1/2) ~ I(GroundSF^(1/2)) + I(YearBuilt^(1/2)) + I(BasementSF^(1/2)) + I(YearRemodel^(1/2)) + I(BasementFinSF^(1/2)) + I(GarageSF^(1/2)) + I(LotArea^(1/2)) + I(Fireplaces^2) + I(LotFrontage^2) + I(Bedroom^2) + I(ScreenPorchSF^2) + I(EnclosedPorchSF^(1/2)) + I(HalfBath^(1/2)), data = AmesTrain2)
plot(mod5)
AmesTrain2$PorchSF = AmesTrain2$ScreenPorchSF + AmesTrain2$EnclosedPorchSF
mod6=lm(Price ~ GroundSF + YearBuilt + BasementSF + YearRemodel +
BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage +
Bedroom + PorchSF + HalfBath, data = AmesTrain2)
plot(mod6)
mod7=lm(sqrt(Price)~ GroundSF + YearBuilt + BasementSF + YearRemodel +
BasementFinSF + GarageSF + LotArea + Fireplaces + LotFrontage +
Bedroom + PorchSF + HalfBath, data = AmesTrain2)
plot(mod7)
As you can see, mod3 performed the best in terms of meeting the conditions of a simple linear model. We’ll discuss this in further detail below.
Part 4
Residual Analysis for Fancier Model
plot(mod3)
hist(mod3$residuals)
Looking at the residuals vs fitted values plot, the linearity condition seems to be met. There appears to be little to no curvature of the residuals, and the residuals hover around zero, suggesting that the line seems to fit the data and no other nonlinear pattern is clear
The zero mean condition seems to be met as well, as the residuals are spread around the zero line in the residuals vs. fitted value plot. The horizontal zero line in the residuals vs. fitted plot seems to pass through the center of the data.
The constant variance condition also appears to be met, as there appears to be equal variance of residuals at all fitted values in the residuals vs. fitted values plot.
Looking at the normal q-q plot and the histogram, the independence condition appears to be met. In the normal q-q plot, the points generally follow the line. The two ends of plot trail off a little, but not by a large amount. The histogram has a general bell curve and does not have a strong skew (it may appear skewed left at first but it is really centered at 0 upon further investigation), further suggesting that the independence condition is met.
Our residual analysis supports that our mod3 fits the conditions of a simple linear model.
Part 5
As our residual analysis suggested that our model fits the conditions of a simple linear model, we made no changes to our model.
newx= data.frame(HouseStyle="2Story", TotalRooms = 9, YearBuilt= 1995, YearRemodel = 2003, LotArea= 11060, LotConfig = "Corner", LotFrontage= 90, Quality = 7, Condition = 5, ExteriorQ = "Gd", ExteriorC = "Gd", Foundation = "PConc", BasementFinSF=0, BasementSF= 1150, BasementHt = "Ex", BasementFin = "Unf", BasementFBath = 0, BasementHBath= 0, Heating = "GasA", HeatingQC= "Ex", CentralAir = "Y", GroundSF = 2314, FirstSF= 1164, SecondSF= 1150, Bedroom = 3, FullBath =2, HalfBath = 1, Fireplaces = 1, GarageCars=2, GarageSF= 502, GarageType= "BuiltIn", GarageQ= "TA" , GarageC= "TA", OpenPorchSF= 274, ScreenPorchSF= 0, EnclosedPorchSF=0 )
predict.lm(mod3, newx, interval="prediction", level=.95)## fit lwr upr
## 1 5.547478 5.211403 5.883553exp(5.211403)## [1] 183.3511exp(5.883553)## [1] 359.0828The 95% Prediction Interval for the mean price in thousands at with the conditions listed is (183.3511, 359.0828). This means that we can be 95% confident that the price of an individual house that meets the listed criteria is between 183,351.10 and 359,082.80 dollars.